Smart desk with gesture detection and control features

ABSTRACT

A smart desk having one or more touch displays is provided. The smart desk can be programmed with one or more touch or touchless gestures for executing computer commands or manipulating content displayed on the one or more touch displays. Touchless gestures can be executed by the smart desk by tracking a user&#39;s movements with a motion sensor. The touch gestures can instruct the smart desk to transfer content between the one or more touch displays, expand content on the one or more touch displays, close a tab or a window, minimize a window, and/or switch between applications. The touchless gestures can include copy, paste, scroll, screen shot, page turn, and/or zoom in or out.

BACKGROUND Field of the Disclosure

The present disclosure relates to a smart desk. More particularly, thepresent disclosure relates to a smart desk with gesture detection andcontrol features.

Related Art

Current solutions for workspaces include a desk and a personal computer.The personal computer is usually a desktop computer which is connectedto one or more monitors, a keyboard, a mouse, and speakers. The personalcomputer can also be an all-in-one device including a single monitorwhich also houses a memory, processor, motherboard and other standardcomputer components.

The issue with these workspaces is that they require a user to alwaysinteract with a physical keyboard and mouse to perform all user commandsand functions with respect to any tasks performed on a computer. Forexample, if a user of a computer device has to read a long documentwhich requires scrolling or paging through the document, the user musteither always have his or her hands on the keyboard or mouse and/or mustbe constantly adjusting his or her sitting position to reach thekeyboard or mouse to scroll or page through the document. As a result, auser is unable to sit back in his or her chair and read a document andperform basic user tasks such as zooming in and out, scrolling up anddown, and turning pages.

Another issue with current workspaces is that using touch screens isdifficult. Because desks are not designed for touch screen monitors andworkstations, a touchscreen is normally placed at a distance far fromwhere a user sits so that it is very inconvenient for the user tointeract with the touchscreen. Moreover, these touchscreens only allowbasic movement of content items on the computer screen such as placing acursor or moving and dragging a window. There are no gestures to alloweasy manipulation of content items. Further, with respect to gestures,current solutions only exists at the application level and therefore thegestures only work for a certain application. When using anotherapplication, the gestures may change or there may be no gestures at all.

Yet another issue with current workspaces is that a physical keyboard isrequired. This requires extra space and can lead to carpal tunnelsyndrome. Virtual keyboards can mitigate these issues but the currentsolutions for virtual keyboards are not effective in allowing the userto type quickly and normally as a physical keyboard.

What is needed therefore is a smart desk with gesture motion and touchcontrols that allows a user to perform computer commands to manipulatecontent items without interacting with a mouse or a keyboard. What isalso needed is a smart desk that is immersive and allows a user tomanipulate touchscreens easily. What is also needed is a smart desk witha virtual keyboard that allows a user to type as fast as a physicalkeyboard.

SUMMARY

A smart desk having one or more touch displays is provided. The smartdesk can be programmed with one or more touch or touchless gestures forexecuting computer commands or manipulating content displayed on the oneor more touch displays. Touchless gestures can be executed by the smartdesk by tracking a user's movements with a motion sensor. The touchgestures can instruct the smart desk to transfer content between the oneor more touch displays, expand content on the one or more touchdisplays, close a tab or a window, minimize a window and/or switchbetween applications. The touchless gestures can include copy, paste,scroll, screen shot, page turn, and/or zoom in or out.

A number of different touch gestures can be programmed to be recognizedby the smart desk and can be assigned to perform any computer function.For example, one gesture can include a touch and hold with a firstfinger and a tap with a second finger to the right of the first finger.Another gesture can include a touch and hold with a first finger and atap with a second finger to the left of the first finger. Yet anothergesture can be a touch and hold with two fingers and a tap with a thirdfinger to the right of the first and second fingers. Similarly, anothergesture can be a touch and hold with two fingers and a tap with a thirdfinger to the left of the first and second fingers. Yet another gesturecan be a touch and hold with three fingers and a tap with a fourthfinger to the right of the three fingers. Similarly, another gesture canbe a touch and hold with three fingers and a tap with a fourth finger tothe left of the three fingers. These gestures can be assigned (by a useror programmer) to perform any of the touch and touchless gesturesdescribed above and below.

As noted above, the smart desk can include a motion sensor for trackinga user's movement to detect touchless gestures to perform computercommands. A grab gesture can be indicative of a user's desire to copyselected content and a release or throw gesture can be indicative of theuser's desire to paste the copied content. A gesture where a user holdsand snaps an imagery camera can be indicative of the user's desire totake a screen shot of the content being displayed on the smart desk. Agesture where a user moves one or more fingers up and down can beindicative of the user's desire to scroll displayed content. A gesturewhere a user moves his or her hand backwards and forwards can beindicative of the user's desire to turn pages forward or backward on adocument being displayed on the smart desk. A gesture where a user moveshis or her hands apart can be indicative of the user's desire to zoom inand a gesture where the user moves his or her hands together can beindicative of the user's desire to zoom out.

A smart desk having a first touch display and a second touch display isprovided. The smart desk includes a processor and a memory havingcomputer code stored thereon which, when executed, causes the processorto perform the following functions. First, the smart desk can detect agesture at the first touch display. The smart desk can determine whetherthe gesture is indicative of a user's desire to transfer contentdisplayed on the first touch display to the second touch display. Thesmart desk can cause the content to be transferred from the first touchdisplay to the second touch display if the gesture is indicative of theuser's desire to transfer the content. The gesture can include a firsttouch user input for selecting the content and a second touch user inputfor transferring the selected content from the first touch display tothe second touch display.

A method is also provided. The method includes detecting a gesture at afirst touch display. The gesture includes a first touch user input forselecting content displayed on the first touch display and a secondtouch user input for indicating a desire to transfer the selectedcontent from the first touch display to a second touch display. Themethod includes determining whether the gesture is indicative of auser's desire to transfer content displayed on the first touch displayto the second touch display. The method includes transferring thecontent from the first touch display to the second touch display if thegesture is indicative of the user's desire to transfer the content.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of the disclosure will be apparent from thefollowing Detailed Description, taken in connection with theaccompanying drawings, in which:

FIG. 1 is a perspective view of an example of a smart desk of thepresent disclosure;

FIG. 2 is a flowchart illustrating processing steps for moving contentfrom a first touch display to a second touch display when the secondtouch display is positioned to the left of the first touch display;

FIG. 3 is a flowchart illustrating processing steps for moving contentfrom the first touch display to the second touch display when the secondtouch display is positioned to the right of the first touch display;

FIG. 4 is a flowchart illustrating processing steps for expandingcontent to the left half of a touch display currently displaying thecontent;

FIG. 5 is a flowchart illustrating processing steps for expandingcontent to the right half of a touch display currently displaying thecontent;

FIG. 6A is a flowchart illustrating processing steps for closing anactive tab within a window;

FIG. 6B is a flowchart illustrating processing steps for minimizing awindow;

FIG. 7 is a flowchart illustrating processing steps for closing anactive window;

FIG. 8 is a flowchart illustrating processing steps for switchingbetween applications;

FIG. 9 is a flowchart illustrating processing steps for copying selectedcontent;

FIG. 10 is a flowchart illustrating processing steps for pasting copiedcontent;

FIG. 11 is a flowchart illustrating processing steps for taking ascreenshot;

FIG. 12 is a flowchart illustrating processing steps for scrolling;

FIG. 13 is a flowchart illustrating processing steps for turning a page;

FIG. 14 is a flowchart illustrating processing steps for zooming in;

FIG. 15 is a flowchart illustrating processing steps for zooming out;

FIG. 16 is a drawing illustrating a haptic response system for keys of avirtual keyboard of the smart desk; and

FIG. 17 is a diagram illustrating example hardware and softwarecomponents of the smart desk.

DETAILED DESCRIPTION

The present disclosure relates to a smart desk with gesture detectionand control features, as discussed in detail below in connection withFIGS. 1-17.

FIG. 1 is a perspective view of an example of a smart desk 2 of thepresent disclosure. The smart desk 2 includes a display housing 4, asurface housing 6, one or more support members 8, and one or more legs10. The display housing 4 can be attached to the surface housing 6 by adisplay support member 12. The one or more support members 8 can beattached to the surface housing 6 and the one or more legs 10. The oneor more support members 8 and the one or more legs 10 bear the weight ofthe smart desk 2. The one or more legs 10 can be elongated in length andcan be positioned along substantially the same angle as the surfacehousing 6 to provide a greater degree of balance and support to thesmart desk 2. The one or more legs 10 can be positioned at an angle toallow a greater amount of space for a user of the smart desk to move hisor feet or chair so as to not interfere with the one or more legs 10.The smart desk 2 can include a personal computer or computer componentsneeded for the personal computer. These components are housed neatlywithin the smart desk 2. The computer components such as the processor,memory, storage, motherboard, etc. can be housed in the display housing4 or the surface housing 6 to provide an integrated clean look and feelfor the user of the smart desk. However, the smart desk 2 is not limitedto any specific location for these computer components which can behoused externally to the smart desk 2. Accordingly, the smart desk 2 isa single integrated device that functions as a desk and a personalcomputer.

The display housing 4 can include a first touch display 14A, a secondtouch display 14B, and a third touch display 14C. As shown in FIG. 1,the first touch display 14A can be positioned to the left of the secondtouch display 14B and the third touch display 14C can be positioned tothe right of the second touch display 14B. The second touch display 14Bcan be positioned so that the head and neck of a user of the smart desk2 is facing directly forward and the eyes of the user can look directlyforward to view the second touch display 14B. The first touch display14A and the third touch display 14C can be positioned at an anglerelative to the second touch display 14B so that the user of the smartdesk 2 is immersed in the smart desk 2. The angle can be the same angleas the back edge of the surface housing to save space and provide abetter appearance for the smart desk 2. The first touch display 14A, thesecond touch display 14B and the third touch display 14C can beconfigured as touch screens so that a user can manipulate content on thescreens by using his or her finger or a smart stylus or similar device.The smart desk 2 is not limited to any particular arrangement of thetouch displays as the first touch display 14A, the second touch display14B, and the third touch display 14C can be positioned in anyarrangement. The smart desk 2 is not limited to any specific number oftouch displays and can include two or more touch displays. The smartdesk 2 can also be designed as one touch display where software of thesmart desk can recognize one or more portions of the touch display assections where each section can function as a separate “touch display”as described in greater detail below.

The display housing 4 can also include a plurality of wedges 16. Theplurality of wedges 16 can be positioned between the second touchdisplay 14B and the first touch display 14A, and the second touchdisplay 14B and the third touch display 14C. The plurality of wedges 16allows the first touch display 14A and the third touch display 14C to bepositioned at the proper angle relative to the second touch display 14Bto provide a immersive experience for the user of the smart desk 2. Theplurality of wedges 16 also allows the first touch display 14A and thethird touch display 14C to be positioned at the proper tilt to providecomfortable viewing angle for the user of the smart desk.

The plurality of wedges 16 can also house other peripherals of the smartdesk 2. For example, the plurality of wedges can house a plurality ofspeakers 18 for outputting sound produced by the smart desk 2. Theplurality of wedges can also house a smart stylus 20 for manipulatingcontent on the touch displays of the smart desk 2. The smart stylus 20can also be used to perform the gestures of the smart desk 2 with thesmart stylus 20 functioning as a finger of a user as will be describedin greater detail below. The plurality of wedges 16 can also house ascanner sensor 22 for scanning documents placed on the surface housing6, as will be explained in greater detail below. Additionally, otherperipherals can be included such as a chargeable headphone, amicrophone, and/or a web camera.

The surface housing 6 can include a virtual keyboard 24. The user of thesmart desk 2 can use the virtual keyboard 24 in the same way as a normalphysical keyboard connected to a personal computer. The virtual keyboard24 can have the same keys as a standard keyboard. Alternatively, thesmart desk 2 can include a physical keyboard which can be connected tothe smart desk 2 via Bluetooth or other wireless means. The physicalkeyboard can also be connected via a wired USB connection if desired. Amouse can also be used with the smart desk 2, but as will be explainedin greater detail below, the smart desk 2 includes a number of gesturesto manipulate content which can obviate the need for a mouse. Moreover,the smart desk 2 includes touch displays which also obviate the need fora mouse. However, if a mouse is desired, it can be connected to thesmart desk 2. Alternatively, the smart desk 2 can include a digitalmouse trackpad 26 built into the surface housing 6. The surface housing6 can also include a mouse trackpad 26 built into and integrated withthe surface housing 6. The surface housing 6 can also include a wirelesscharger 27 for charging a smartphone, smart watch, or any other devicecapable of being charged by inductive charging means. The wirelesscharger 27 can conform to the Qi standard or any other wireless chargingstandards. The charge level of the devices being charged by the wirelesscharger 27 can be displayed on the touch displays of the smart desk 2.The wireless charger 27 can be built into the surface housing 6 so thata flat integrated surface is presented to a user. The smart desk 2 cansynch with a user's smart phone via Bluetooth to provide integratedaccess calling features such as a display on a touchscreen of the smartdesk 2 to show a caller's identification.

A user of the smart desk 2 can use their finger or the smart stylus 20for writing on any of the touch displays of the smart desk 2. The secondtouch display 14B can be lowered and/or removed from the display housing4 so that a user can write on it like a notepad on the surface housing6.

The surface housing 6 can also include a motion sensor 28 for detectingmovement of a user of the smart desk 28. The motion sensor 28 can be anysensor known in the art for detecting movement of the user of the smartdesk 2 such as a Leap Sensor. The motion sensor 28 can be placed at theedge of the surface housing 6 between the virtual keyboard 24 and theuser of the smart desk 2 so that the motion sensor 28 is proximal to theuser. The smart desk 2 is not limited to any specific location of themotion sensor 28 which can be placed in any desired location on thesmart desk 2 to adequately detect motion of the user of the smart desk2. The motion sensor 28 can even be placed on another location outsideof the smart desk 2. The smart desk 2 also includes a scan area 30 whichis a portion of the surface housing 6. The scan area 30 allows a user toplace a document to be scanned by the scanner sensor 22. The smart desk2 can provide software for allowing the user to initiate the scanprocess and to present the user with the document on one of thetouchscreens of the smart desk 2. The smart desk 2 can alsoautomatically apply optical character recognition to the document sothat the user can search the document once it is scanned.

FIGS. 2-8 will now be discussed in greater detail. Each of these figuresdescribes processing steps for executing a touch “gesture” which canperform some computer related task. In particular, a user of the smartdesk 2 performs an action on a touchscreen with the intent of performinga computer operation (e.g., moving a content item to anothertouchscreen). As will be discussed below in connection with FIGS. 2-8,each gesture results in the smart desk 2 executing a particular computeroperation. However, each gesture is not limited or tied to any specificcomputer operation. For example, a user of the smart desk 2 can assignand re-assign a gesture to perform any computer operation describedherein or other computer operations entirely. Accordingly, the smartdesk 2 is modular in that performing a gesture can be assigned to anycomputer task.

FIG. 2 is a flowchart illustrating processing steps 50 for executing agesture for moving a content item from a first touch display displayingthe content item to a second touch display positioned to the left of thefirst touch display. In step 52, the smart desk 2 receives a first userinput of a touch and hold with a first finger on a content item toselect the content item. In step 54, the smart desk 2 receives a seconduser input of a tap with a second finger to the left of the firstfinger. Execution of these steps can be indicative of a user desire totransfer the content item to a touch display positioned to the left ofthe touch displaying currently displaying the content item. Accordingly,in step 56, once this gesture is executed, the smart desk 2 can move thecontent item to a touch display positioned to the left of the touchdisplay currently displaying the content item. By way of example, if acontent item is currently being displayed on the second touch display14B, the user can execute the above gesture to transfer the content itemto the first touch display 14A. Although the process steps 50 requiresthe first and second user inputs to be executed with a first and asecond finger, a plurality of fingers can be used for either of theseinputs. Moreover, the first input does not need to be a touch and holdand can include some other touch input such as a tap. Similarly, thesecond input does not have to be a tap and can be any other touch inputsuch as a touch and hold. The above gesture can also be executed inother finger combinations as discussed herein with respect to performingother tasks. This gesture can also be re-assigned by the user of thesmart desk 2 to perform any of the computer tasks discussed herein.

FIG. 3 is a flowchart illustrating processing steps 60 for executing agesture for moving a content item from a first touch display displayingthe content item to a second touch display positioned to the right ofthe first touch display. In step 62, the smart desk 2 receives a firstuser input of a touch and hold with a first finger on a content item toselect the content item. In step 64, the smart desk 2 receives a seconduser input of a tap with a second finger to the right of the firstfinger. Execution of these steps can be indicative of a user desire totransfer the content item to a touch display positioned to the right ofthe touch display currently displaying the content item. Accordingly, instep 66, once this gesture is executed, the smart desk 2 can move thecontent item to a touch display positioned to the right of the touchdisplay currently displaying the content item. By way of example, if acontent item is currently being displayed on the second touch display14B, the user can execute the above gesture to transfer the content itemto the third touch display 14C. Although the process steps 60 requiresthe first and second user inputs to be performed with a first and asecond finger, a plurality of fingers can be used for either of theseinputs. Moreover, the first input does not need to be a touch and holdand can include some other touch input such as a tap. Similarly, thesecond input does not have to be a tap and can be any other touch inputsuch as a touch and hold. The above gesture can computer tasks discussedherein.

FIG. 4 is a flowchart illustrating processing steps 70 for executing agesture for expanding a content item on the left half of a touch displaycurrently displaying the content item. In step 72, the smart desk 2receives a first user input of a touch and hold with a first finger on acontent item to select the content item. In step 74, the smart desk 2receives a second user input of a tap with a second finger to the leftof the first finger. Execution of these steps can be indicative of auser desire to expand the content item to the left half of the touchdisplay currently displaying the content item. Accordingly, in step 76,once this gesture is executed, the smart desk 2 can expand the contentitem to the left half of the touch display so that content item ismaximized and sized automatically to fill the left half of the touchdisplay. Although the process steps 70 requires the first and seconduser inputs to be done a first and a second finger, a plurality offingers can be used for either of these inputs. Moreover, the firstinput does not need to be performed with a touch and hold input and caninclude some other touch input such as a tap. Similarly, the secondinput does not have to be a tap and can be any other touch input such asa touch and hold. The above gesture can also be executed in other fingercombinations as discussed herein with respect to performing other tasks.This gesture can also be re-assigned by the user of the smart desk 2 toperform any of the computer tasks discussed herein.

FIG. 5 is a flowchart illustrating processing steps 80 for executing agesture for expanding a content item on the right half of a touchdisplay currently displaying the content item. In step 82, the smartdesk 2 receives a first user input of a touch and hold with a firstfinger on a content item to select the content item. In step 84, thesmart desk 2 receives a second user input of a tap with a second fingerto the right of the first finger. Execution of these steps can beindicative of a user desire to expand the content item to the right halfof the touch display currently displaying the content item. Accordingly,in step 86, once this gesture is executed, the smart desk 2 can expandthe content item to the right half of the touch display so that contentitem is maximized and sized automatically to fill the right half of thetouch display. Although the process steps 80 requires the first andsecond user inputs to be performed with a first and a second finger, aplurality of fingers can be used for either of these inputs. Moreover,the first input does not need to be a touch and hold input and caninclude some other touch input such as a tap. Similarly, the secondinput does not have to be a tap and can be any other touch input such asa touch and hold. The above gesture can also be executed in other fingercombinations as discussed herein with respect to performing other tasks.This gesture can also be re-assigned by the user of the smart desk 2 toperform any of the computer tasks discussed herein.

As can be seen in FIGS. 2-5, the gesture to transfer a content item fromone touch display to another touch display is the same as the gesture toexpand a window to either a right half or left half of a screen. If itis desired for the smart desk 2 to have both of these gestures, then thesmart desk 2 can have an additional requirement for differentiatingbetween the gestures. One example difference can be the distance betweenthe fingers of a user when selecting content in the first user input andexecuting the gesture in the second user input. For example, a gestureto transfer content items between touch displays can be executed if thesecond user input of a tap of a second finger occurs at a firstpredetermined distance from the first input of a touch and hold. Thispredetermined distance can be 2 inches by way of example. The gesture toexpand a window to the right or left of the current touch display canthen be executed if the second user input of a tap of a second fingeroccurs at a second predetermined distance from the first input of atouch and hold. This predetermined distance can be 1 inch by way ofexample. The above distances can be ranges of a distance. For examplethe first predetermined distance can be between 1.5 and 2.5 inches andthe second predetermined distance can be 0.5 and 1.5 inches.

FIG. 6A is a flowchart illustrating processing steps 90 for executing agesture for closing an active tab within a window. For example, a webbrowser can have multiple tabs where different websites are opened, andthis gesture, for example, can close the current tab within a webbrowser. In step 92, the smart desk 2 receives a first user input of atouch and hold with a first and second finger on a content item toselect the content item (e.g., a tab within an active window). In step94, the smart desk 2 receives a second user input of a tap with a thirdfinger to the left of the first and second finger. Execution of thesesteps can be indicative of a user desire to close a current tab within awindow. Accordingly, in step 95, once this gesture is executed, thesmart desk 2 can close the current tab within an active window. Thisgesture obviates the need for a user to use a mouse and click a small“x” that is usually located on a tab of a browser or some other window.Although the process steps 90 requires the first user input to use twofingers and the second user input to use one finger, any number offingers can be used to execute this gesture. Moreover, the first userinput does not need to be a touch and hold and can include some othertouch input such as a tap. Similarly, the second user input does nothave to be a tap and can be any other touch input such as a touch andhold. The above gesture can also be executed in other fingercombinations as discussed herein with respect to performing other tasks.This gesture can also be re-assigned by the user of the smart desk 2 toperform any of the computer tasks discussed herein.

Optionally, the gesture described in FIG. 6A can be assigned to minimizea window. Accordingly, FIG. 6B is a flowchart illustrating processingsteps 96 for executing a gesture for minimizing a window. This alternategesture can obviate the need for a user to click on the small minimizeicon on the top right of most windows. In step 97, the smart desk 2receives a first user input of a touch and hold with a first and secondfinger on a window to select the window to be minimized. In step 98, thesmart desk 2 receives a second user input of a tap with a third fingerto the left of the first and second finger. Execution of these steps canbe indicative of a user desire to minimize the window. Accordingly, instep 99, once this gesture is executed, the smart desk 2 can minimizethe window. Although the process steps 96 requires the first user inputto use two fingers and the second user input to use one finger, anynumber of fingers can be used to execute this gesture. Moreover, thefirst user input does not need to be a touch and hold and can includesome other touch input such as a tap. Similarly, the second user inputdoes not have to be a tap and can be any other touch input such as atouch and hold. The above gesture can also be executed in other fingercombinations as discussed herein with respect to performing other tasks.This gesture can also be re-assigned by the user of the smart desk 2 toperform any of the computer tasks discussed herein.

FIG. 7 is a flowchart illustrating processing steps 100 for executing agesture for closing an active window. In step 102, the smart desk 2receives a first user input of a touch and hold with a first and secondfinger on a content item to select the content item (e.g., an activewindow). In step 104, the smart desk 2 receives a second user input of atap with a third finger to the right of the first and second finger.Execution of these steps can be indicative of a user desire to close anactive window. Accordingly, in step 106, once this gesture is executed,the smart desk 2 can close the current window. This gesture obviates theneed for a user to use a mouse and click a small “x” that is usuallylocated on a window. Although the process steps 100 requires the firstuser input to use two fingers and the second user input to use onefinger, any number of fingers can be used to execute this gesture.Moreover, the first user input does not need to be a touch and hold andcan include some other touch input such as a tap. Similarly, the seconduser input does not have to be a tap and can be any other touch inputsuch as a touch and hold. The above gesture can also be executed inother finger combinations as discussed herein with respect to performingother tasks. This gesture can also be re-assigned by the user of thesmart desk 2 to perform any of the computer tasks discussed herein.

FIG. 8 is a flowchart illustrating processing steps 110 for executing agesture for switching between applications running on an operatingsystem of the smart desk 2. In step 112, the smart desk 2 receives afirst user input of a touch and hold with a first, second finger, andthird finger anywhere on the screen. In step 114, the smart desk 2receives a second user input of a tap with a fourth finger to the leftof the first, second and third fingers. Execution of these steps can beindicative of a user desire to switch between applications. Accordingly,in step 116, once this gesture is executed, the smart desk 2 can switchbetween applications. Although the process steps 110 requires the firstuser input to use three fingers and the second user input to use onefinger, any number of fingers can be used to execute this gesture.Moreover, the first user input does not need to be a touch and hold andcan include some other touch input such as a tap. Similarly, the seconduser input does not have to be a tap and can be any other touch inputsuch as a touch and hold. The above gesture can also be executed inother finger combinations as discussed herein with respect to performingother tasks. This gesture can also be re-assigned by the user of thesmart desk 2 to perform any of the computer tasks discussed herein.

As noted above, the gestures described above in connection with FIGS.2-8 are not limited to the computer operation performed at the end ofthe process steps in FIGS. 2-8. Each gesture can correspond to anycomputer task described herein or any other computer task that can beperformed. For example, with respect to the gesture described in FIGS.2-3 where a user touches and holds with a first finger and taps to theleft/right with a second finger (at a first predetermined distance awayfrom the first finger), this action can be assigned to perform any othertask described herein such as expanding an active window, minimizing awindow, closing a tab or a window, switching between applications, orsome other computer task. Moreover, with respect to the gesturedescribed in FIGS. 4-5 where a user touches and holds with a firstfinger and taps to the left/right with a second finger (at a secondpredetermined distance away from the first finger), this action can beassigned to perform any other task described herein such moving contentbetween touchscreens, minimizing a window, closing a tab or a window,switching between applications, or some other computer task. Further,with respect to the gesture described in FIG. 6A where a user touchesand holds a first and second finger on an active tab within a window (toclose the tab) and then taps with a third finger to the left of thefirst and second finger, this action can be assigned to perform anyother task described herein such minimizing a window, moving contentbetween touchscreens, expanding an active window left or right, closingan active window, switching between applications, or some other computertask. Moreover, with respect to the gesture described in FIG. 6B where auser touches and holds a first and second finger on a window and thentaps with a third finger to the left of the first and second finger tominimize the selected window, this action can be assigned to perform anyother task described herein such closing an active tab, moving contentbetween touchscreens, expanding an active window left or right, closingan active window, switching between applications, or some other computertask. Still further, with respect to the gesture described in FIG. 7where a user touches and holds a first and second finger on an activewindow (to closed the window) and then taps with a third finger to theright of the first and second finger, this action can be assigned toperform any other task described herein such moving content betweentouchscreens, expanding an active window left or right, closing a tabwithin an active window, switching between applications, or some othercomputer task. Additionally, with respect to the gesture described inFIG. 8 where a user touches and holds a first, second and third fingeron the screen (to switch between applications) and then taps with athird finger to the left of the first, second and third fingers, thisaction can be assigned to perform any other task described herein suchmoving content between touchscreens, expanding an active window left orright, closing an active window, closing a tab within an active window,or some other computer task.

FIGS. 9-15 will now be described in greater detail. These figuresillustrate processing steps for touchless gestures to execute computercommands. These touchless gestures do not require a user to interactwith a touchscreen on the smart desk 2. Rather, a user of the smart desk2 can make gestures while sitting back in his or her chair and the smartdesk 2 can always track the user with the motion sensor 28. Upon apositive determination that a user is performing a gesture to execute acomputer command, the smart desk 2 can execute the command as will bedescribed in greater detail below.

FIG. 9 is a flowchart illustrating processing steps 120 for executing agesture to copy selected content. In step 122, the smart desk 2 canreceive a first user input of a first finger extended on a first handand a grab gesture with a second hand. Alternatively, a user of thesmart desk 2 can make a grab gesture with a first hand without requiringa finger to be extended. The grab gesture can be executed with all fivefingers of a hand where the fingers are initially extended and then forma closed first as if the user is actually grabbing the selected content.The smart desk 2 can make a “crunch” or a sound of a paper crumpling upor some other sound to provide feedback to the user that the smart desk2 successfully copied the selected content.

FIG. 10 is a flowchart illustrating processing steps 130 for executing agesture to paste copied content. In step 132, the smart desk 2 canreceive a first user input of a first finger extended on a first handand a release gesture with a second hand. Alternatively, a user of thesmart desk 2 can make a release gesture with a first hand withoutrequiring a finger to be extended. The release gesture can be executedwith all five fingers starting as a closed first and then extending thefingers open as if the user is releasing copied content or “throwing”the copied content. The smart desk 2 can make a sound to providefeedback to the user that the paste command has been executed. Anexample of this can be a “slap” or similar sound.

The copy gesture and paste gesture can be programmed to be complimentarygestures. For example, if a user “grabs” selected content to copy it,the user can then release his or her first to paste the selected contentin one smooth motion. The smart desk 2 can also allow the user to “drop”the copied content by releasing his or her fist.

FIG. 11 is a flowchart illustrating processing steps 140 for executing agesture to take a screenshot by capturing all screen content and copyingit to the clipboard. In step 142, the smart desk 2 receives a first userinput of a thumb and forefinger on each hand positioned as if the useris holding an imaginary camera. Alternatively, other fingers can be usedto make a shape of an imaginary camera. In step 144, the smart desk 2receives a second user input of the forefingers moving toward the thumbsas if the user snapping a picture with the imaginary camera. As notedabove, other fingers or motions can be used to indicate that animaginary camera is being snapped. One finger to make the snappingmotion can also be used. In step 146, the smart desk takes a screenshotby capturing all screen content and copying it to the clipboard.

FIG. 12 is a flowchart illustrating processing steps 150 for executing agesture to scroll content on the smart desk. In step 152, the smart desk2 receives a first user input of a first and second finger extended fromone hand moving in a up or down direction. Alternatively, the smart desk2 can require any number of fingers to execute this gesture. In step154, the smart desk scrolls content in the direction of the movement ofthe first and second fingers. This gesture functions as if a user isusing a wheel of a mouse or scroll bar on an application to move up ordown in a document. However, with this touchless gesture, a user can sitback in his or her chair and scroll through a document without having tointeract with a keyboard or mouse. Accordingly, this gesture does notrequire a user to interact with the touchscreens of the smart desk 2.The direction of the scroll based on upward or downward movement can beset by the user based on preference. The scroll gesture can move upwardor downward in a fluid way such that the fingers can move up or downcontemporaneously and smoothly without having to reset his or herfingers.

FIG. 13 is a flowchart illustrating processing steps 160 for executing agesture to turn a page in a document. In step 162, the smart deskreceives a first user input of a first hand opened and swiping in eitherthe right or left direction as if the user is swiping pages of a book.Any number of fingers can be extended to execute this gesture. In step164, the smart desk 2 turns a page of a document in the direction of themovement of the first hand. The smart desk 2 can allow a user to set hisor her preference as to which direction turns a page forward andbackward. The smart desk 2 can be programmed to turn a page forwardwhere the user moves his hand from right to left and to turn a pagebackward where the user moves his hand from left to right as if the useris actually turning a page in a book. As noted above, the user canchange this preference if desired.

The page forward and backward gestures can be complimentary gestures inthat a user can execute these motions contemporaneously without havingto reset his or her hands. In essence these gestures can provide a senseof fluidity for a user to page forward and backward in one constant setof motions without interruption.

FIG. 14 is a flowchart illustrating processing steps 170 for zooming inon an active window. In step 172, the smart desk 2 receives a first userinput of both hands of a user wide open and moving apart. In step 174,the smart desk 2 zooms in on the active window. This gesture can beperformed by other similar mechanisms indicating that a user wants tozoom in on a document. For example, the user can use two fingers andmove them apart instead of wide open hands.

FIG. 15 is a flowchart illustrating processing steps 180 for zooming outon an active window. In step 182, the smart desk 2 receives a first userinput of both hands of a user wide open and moving together. In step184, the smart desk 2 zooms out on the active window. This gesture canbe performed by other similar mechanisms indicating that a user wants tozoom out on a document. For example, the user can use two fingers andmove them closer together instead of wide open hands.

The zoom in and out gestures can be complimentary gestures in that auser can execute these motions contemporaneously without having to resethis or her hands. In essence these gestures can provide a sense offluidity for a user to zoom in and out in one fluid motion to achievethe desired zoom.

FIG. 16 is a drawing which illustrates a haptic response system 190 of akey 192 of the virtual keyboard 24 of the smart desk 2. As can be seenin FIG. 16, the “J” key 192 is shown as an example, but each key on thevirtual keyboard 24 can be similarly designed. Each key on the virtualkeyboard 24 can be designed with an enhanced haptic feedback systemwhere pressing the key closer to the target (e.g., the letter J as shownin FIG. 16) provides a greater degree of haptic feedback. For example,if the user presses the key 192 within a first area 194, the smart desk2 provide a first haptic response. If the user presses the key 192within a second area 196 but also outside the first area 194, the smartdesk 2 provides a second haptic response. If the user presses within athird area 198 but outside of the second area 196 and the first area194, the smart desk 2 provides a third haptic response. If the userpresses down outside of the third area 198, then no haptic response canbe provided by the smart desk 2. The closer the user's typing stroke isto the key 192, the greater the haptic response can be. For example, thefirst haptic response can be a 100 percent, the second haptic responsecan be 60 percent and the third haptic response can be 30 percent. Thisprovides a user with enhanced feedback as to the accuracy of thekeystrokes and the location in which the keystrokes are occurring whichallows the user to use a virtual keyboard as if it is a physicalkeyboard, and type at the speed the user types on the physical keyboard.

The virtual keyboard 24 of the smart desk 2 can include a first sensingmechanism for determining where a user presses on the virtual keyboard24. The first sensing mechanism can determine which key a user presseson the virtual keyboard 24. The first sensing mechanism can have theappropriate degree of sensitivity to determine in close proximitywhether a user intended to press a particular key over any of the otheradjacent keys on the virtual keyboard 24. The first sensing mechanismcan use infrared sensors, or any other sensors known in the art. Thevirtual keyboard 24 can also include a second sensing mechanism todetermine whether a user intended to press a key or whether the user ismerely resting his or her finger(s) on the key. Accordingly, the virtualkeyboard 24 can include a plurality of pressure sensors to make thisdetermination.

All of the above touch and touchless gestures described in connectionwith FIGS. 2-15 can be programmed on the operating system (e.g., Windows10) running on the smart desk 2. This allows the gestures to beperformed and executed on any application seamlessly.

FIG. 17 is a diagram illustrating example hardware and softwarecomponents of the smart desk 2. The smart desk 2 could include a storagedevice 204, a network interface 218, a communications bus 210, a centralprocessing unit (CPU) (microprocessor) 212, a random access memory (RAM)214, and one or more input devices 216, such as a keyboard, mouse, etc.The smart desk 2 could also include a touch displays (e.g., liquidcrystal display (LCD), cathode ray tube (CRT), etc.) as described ingreater detail above. The storage device 204 could comprise anysuitable, computer-readable storage medium such as disk, non-volatilememory (e.g., read-only memory (ROM), eraseable programmable ROM(EPROM), electrically-eraseable programmable ROM (EEPROM), flash memory,field-programmable gate array (FPGA), etc.). The functionality providedby the present disclosure could be provided by a gesture control engine206, which could be embodied as computer-readable program code stored onthe storage device 204 and executed by the CPU 212 using any suitable,high or low level computing language, such as Python, Java, C, C++, C #,.NET, MATLAB, etc. The network interface 218 could include an Ethernetnetwork interface device, a wireless network interface device, or anyother suitable device which permits the smart desk 2 to communicate viathe network. The CPU 212 could include any suitable single- ormultiple-core microprocessor of any suitable architecture that iscapable of implementing and running the gesture control engine 206(e.g., Intel processor). The random access memory 214 could include anysuitable, high-speed, random access memory typical of most moderncomputers, such as dynamic RAM (DRAM), etc.

Having thus described the system and method in detail, it is to beunderstood that the foregoing description is not intended to limit thespirit or scope thereof. It will be understood that the embodiments ofthe present disclosure described herein are merely exemplary and that aperson skilled in the art may make any variations and modificationwithout departing from the spirit and scope of the disclosure. All suchvariations and modifications, including those discussed above, areintended to be included within the scope of the disclosure.

What is claimed is:
 1. A smart desk comprising: a first touch displayand a second touch display; one or more processors; and memory storinginstructions that, when executed by the one or more processors, causethe smart desk to: (i) receive a first touch user input on the firsttouch display; (ii) determine, based on the first touch user inputindicating a first touch on the first touch display, a selection of acontent item; (iii) receive a second touch user input on the first touchdisplay; (iv) determine, based on receiving the second touch user inputafter the first touch user input, and based on the second touch beingreceived while the first touch is received, to transfer the content itemfrom the first touch display to the second touch display; and (v)transfer, based on the determination to transfer the content item,output of the content item from the first touch display to the secondtouch display.
 2. The smart desk of claim 1, wherein the first touchuser input includes the first touch on the selected content item with atleast one finger and holding the first touch on the selected contentitem with the at least one finger, and the second touch is a tap with atleast one other finger.
 3. The smart desk of claim 2, wherein when thesecond touch display is positioned to the right of the first touchdisplay, the selected content item is moved to the second touch displaywhen the at least one other finger is tapped to the right of the atleast one finger.
 4. The smart desk of claim 2, wherein when the secondtouch display is positioned to the left of the first display, theselected content item is moved to the second display when the at leastone other finger is tapped to the left of the at least one finger. 5.The smart desk of claim 3 further comprising a third touch displaypositioned to the left of the first touch display, and wherein theselected content item is moved to the third touch display when the atleast one other finger is tapped to the left of the at least one finger.6. The smart desk of claim 4 further comprising a third touch displaypositioned to the right of the first touch display, and wherein theselected content item is moved to the third touch display when the atleast one other finger is tapped to the right of the at least onefinger.
 7. The smart desk of claim 1, wherein the selected content itemis transferred from the first touch display to the second touch displaywhen the second touch user input occurs between a first predetermineddistance and a second predetermined distance from the first touch userinput.
 8. The smart desk of claim 7, wherein the selected content itemis not transferred if the second touch user input does not occur betweenthe first predetermined distance and the second predetermined distancefrom the first touch user input.
 9. The smart desk of claim 1, whereinthe instructions are programmed on an operating system level so that theinstructions can be executed on any application running on the operatingsystem of the smart desk.
 10. A method comprising the steps of:receiving a first touch user input on a first touch display;determining, based on the first touch user input indicating a firsttouch on the first touch display, a selection of a content item;receiving a second touch user input on the first touch display;determining, based on receiving the second touch user input after thefirst touch user input, and based on the second touch being receivedwhile the first touch is received, to transfer the content item from thefirst touch display to the second touch display; and transferring, basedon the determination to transfer the content item, output of the contentitem from the first touch display to the second touch display.
 11. Themethod of claim 10, further comprising the step of determining whetherthe first touch user input includes the first touch on the selectedcontent item with at least one finger and holding the first touch on theselected content item with the at least one finger, and whether thesecond touch is a tap with at least one other finger.
 12. The method ofclaim 11, further comprising the step of transferring the selectedcontent item from the first touch display to the second touch displaywhen the second touch display is positioned to the right of the firsttouch display and when the at least one other finger is tapped to theright of the at least one finger.
 13. The method of claim 11, furthercomprising the step of transferring the selected content item from thefirst touch display to the second touch display when the second touchdisplay is positioned to the left of the first touch display and whenthe at least one other finger is tapped to the left of the at least onefinger.
 14. The method of claim 12 further comprising the step oftransferring the selected content item from the first touch display to athird touch display when the third touch display is positioned to theleft of the first touch display and when the at least one other fingeris tapped to the left of the at least one finger.
 15. The method ofclaim 13 further comprising the step of transferring the selectedcontent item from the first touch display to a third touch display whenthe third touch display is positioned to the right of the first touchdisplay and when the at least one other finger is tapped to the right ofthe at least one finger.
 16. The method of claim 11, further comprisingthe step of transferring the selected content item from the first touchdisplay to the second touch display when the second touch user inputoccurs between a first predetermined distance and a second predetermineddistance from the first touch user input.
 17. The method of claim 16,wherein the selected content item is not transferred if the second touchuser input does not occur between the first predetermined distance andthe second predetermined distance from the first touch user input. 18.The method of claim 11, wherein the steps are programmed on an operatingsystem level so that the steps can be executed on any applicationrunning on the operating system.
 19. A non-transitory computer-readablemedium having thereon computer-executable instructions for performing acomputer-implemented method, wherein the method comprises the steps of:receiving a first touch user input on a first touch display;determining, based on the first touch user input indicating a firsttouch on the first touch display, a selection of a content item;receiving a second touch user input on the first touch display;determining, based on receiving the second touch user input after thefirst touch user input, and based on the second touch being receivedwhile the first touch is received, to transfer the content item from thefirst touch display to the second touch display; and transferring, basedon the determination to transfer the content item, output of the contentitem from the first touch display to the second touch display.
 20. Thenon-transitory computer-readable medium of claim 19, wherein the methodfurther comprises the step of determining whether the first touch userinput includes the first touch on the selected content item with atleast one finger and holding the first touch on the selected contentitem with the at least one finger, and whether the second touch is a tapwith at least one other finger.
 21. The non-transitory computer-readablemedium of claim 20, wherein the method further comprises the step oftransferring the selected content item from the first touch display tothe second touch display when the second touch display is positioned tothe right of the first touch display and when the at least one otherfinger is tapped to the right of the at least one finger.
 22. Thenon-transitory computer-readable medium of claim 20, wherein the methodfurther comprises the step of transferring the selected content itemfrom the first touch display to the second touch display when the secondtouch display is positioned to the left of the first touch display andwhen the at least one other finger is tapped to the left of the at leastone finger.
 23. The non-transitory computer-readable medium of claim 21,wherein the method further comprises the step of transferring theselected content item from the first touch display to a third touchdisplay when the third touch display is positioned to the left of thefirst touch display and when the at least one other finger is tapped tothe left of the at least one finger.
 24. The non-transitorycomputer-readable medium of claim 22, wherein the method furthercomprises the step of transferring the selected content item from thefirst touch display to a third touch display when the third touchdisplay is positioned to the right of the first touch display and whenthe at least one other finger is tapped to the right of the at least onefinger.
 25. The non-transitory computer-readable medium of claim 20,wherein the method further comprises the step of transferring theselected content item from the first touch display to the second touchdisplay when the second touch user input occurs between a firstpredetermined distance and a second predetermined distance from thefirst touch user input.
 26. The non-transitory computer-readable mediumof claim 25, wherein the selected content item is not transferred if thesecond touch user input does not occur between the first predetermineddistance and the second predetermined distance from the first touch userinput.
 27. The non-transitory computer-readable medium of claim 20,wherein the steps are programmed on an operating system level so thatthe steps can be executed on any application running on the operatingsystem.